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This listing of claims will replace all prior versions, and listings, of claims In the application: 

Listing of Ctaims: 
1-15 (cancelled) 

16. (currently amended) The method of Claim 1 95 wherein defining an experiment 
comprises: 

identifying desired objectives for user bohavior;_and 
^IdBTTtifyingTwhich-treatmentsirnayHnfluenceHaser-behavioi^ ■. — — 

objective&r£H=td 

gonarot i ng tho variouo trcQtmonto uoing difforont combinationo of the oontont o l cmonto . 

1 7. (original) The method of Claim 1 6 further comprising: 

defining at least one control variable for the various treatments; and 

assigning a respective level for the control variable for each treatment. 

J 

1 8. (currently amended) The method of Claim 1 9§ wherein conducting the experiment 
comprises: 

selecting at least one treatment for deliveiy to users; 

grouping users into a plurality of segments, each segment comprising users with similar 
behavioral characteristics; and 

specifying a particular segment of users to receive the selected treatment. 

19. (currently amended) An automated computer - implemented method oomprisinq: 

defining an experiment to aauoa uRar reacti o n to various treatments for a set Qf CQHt^m 

- elsmerrtsr ^""" ''^ " ^ "*^' " 

conducting the ex periment over a data networi^: and 

Page 2 - RESPONSE TO OFFICE ACTION DATED AUGUST 10, 2004 
Serial No. 09/648.429 



PA(£4l25'RCVDAT11/9/20044:3OPM [Eastern StandardM^ 

BEST AVAILABLE COPY 



NOV/09/2004/TUE 02:47 PM GANZ LAW 



FAX No. 5032962172 



P. 005 



collecting over the data network obsfirvationr data-retetina to user behavior for each treatment 
Tho mothod of Claim 15 w herein conducting the experiment comprises: 

statistically sampling to specify a plurailty of control groups, each oontrol group 
comprising at least one user; 

specifying a particular treatment to be delivered to the iat least one user In each" control 

group; 

receiving identical requests for content from the respective at least one user in each of 
the control groups; and 

in response to the identical requests, delivering to the at least one user in each control 
group a different treatment. 

20. (original) The method of Claim 19 further comprising observing site-related behavior of 
users receiving the various treatments. 

21 . (currently amended) The method of Claim 1§2 wherein the data nelwori^ comprises the 
Internet 

22. (original) The method of Claim 20 wherein collecting observation data comprises recording 
observed behavior in each control group during tho experiment 

23. (Cun-enUy amended) The method of Claim 1 95 further comprising identifying elements 
of content which potentially influence behavior of users. 

24; (currently arr>ended) An automated system comprising: 
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a content system operable to store content, the content comprising a set of content 
elements; and 

a communication managemant system in communication with the content system and 
operable to define an experiment relating to various treatments for the set of content elements, 
conduct the experiment over a data networic,^ collect over the data network observation data 
relating to user hPhavinrfhr each- treatmen t, wherein the svstem i s configured for statistioallY 
sampling to specify a oluralHv of control oroups. e a c:h control group comprising at least one 
user: 

soecifylnQ a oariicular treatment to be delivered to th e at least one user in eaph, control 

grouni ... . . 

receiving identical requests fnr content from the respective at least one user in e ach of 

the conirol orouDs: and 

In response to the identical reo u ests. delivering to the at least one user in each control 
group a different treatrngpLraftf^ norato at loact ono ocript for ooordinottng tho opo FSlten-ef 
the content Gystom and tho oommunicat l on manogoment ayatom. 

25. (original) The system .of Claim 24 wherein the communication management system is 
operable to identify content elements which may influence user behavior. 

j 26. (cancelled) 

27. (original) TTie system of Claim 24 wherein the communication management system is 
operable to generate a set of experiment rules for allocating treatments during the experiment. 
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28. (original) The system of Claim 24 wherein tine content system ts operable to allocate 
treatments to users according to the experiment rules. 

29. (original) The system of Claim 24 wherein the communication management system is 
operable to group users into a-plurality of segments, each segment comprising users with 
similar behavioral characteristics. 

30. (original) The system of Claim 24 wherein said data networic comprises the Internet 

31 . (original) The system of Claim 24 wherein the communication management system is 
operable to define at least one control variable for the various treatments and to assign a 
respective level for the control variables for each treatment. 

32. (original) The system of Claim 24 wherein the content system and the communication 
management system are connected to the data network. 

33. (original) The system of Claim 24 wherein the cxintent system comprises an allocator 
module operable to support an interface with the communication management system. 

34. (original) The system of Claim 24 wherein the communication management system 
comprises an allocator interface object operable to support an interface with the content system. 

35. (original) The system of Claim 24 further comprising a content provider interfiace operable 
to support an interface between the communication management system and a manager user. 
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36. (Canceled) 

37. (cancelled) 

38. (currently amended) The system of claim 833@ wherein the experiment engine is 
bperable to implement statistical sampling procedures to deliver over the data network the 
various treatments to respective control groups of users. 

39. (currently amended) The system of Claim 833^ wherein the data network comprises 
thjs .Internet. ... 

40-42 (cancelled) 

I 43. (cun-ently amended) A r^mputRr-implemented method for on-Hne experimentation 



defining a plurality of treatments for a set of content elements; 
prov i ding q wcboito on a data notworii ; 

receiving requests for content from user s over a a e ccGaing tho woboito on the data 
network; 



alfocating over the data network a first treatment to each user in the control group; 
allocating over the data network a second treatment to each user not in the control 



group: and 

ooliecting observation data for observed behavior of users in the control group and users 
not in the control group^i-ae^ 
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gonorot i ng at l oaot one soript to coord i nato allocating tho firot c i nd second troatmonto 
and CO Hooting oboorvation data . 

44. (original) The method, of Claim 43 further comprising identifying elements of content which 
may influence user behavior. . . : - 

45. (original) The method ofClaim 43 wherein providing a webstte comprises providing a web 
page at which the set of content elements is available in the form of any of the treatments. 

46. Coriglna!) The method of Clairn 43 v^iarein Vie data nefe^^^ 
47-50 (cancelled) 

51 . (cun^ntly amended) An online experimentation system configured to automatically 
create various treatments for a set of content alaments accorciinQ to systematic rules of 
experimentation that determine a number of alterna tive treatments using various combinations 
of the content elements from the set, and to allocate the treatments to users to test user 
behavior In response to the various treatments, the s^tem being further configured to capture 
data relating to the observed behavior of users relative to the various treatments and to perfomi 
one or more methods comprising: (1^ analyze the d ata to derive a model in form of a 
contingency table: (2) process the data for pathologies. Including at least one of m issing d^t^, 
and structural dependencies, and DrQc e ssina the data to be model ready; (3) maKe inferences 
about some variables that influence user behavi or based on random utilitv theory: and (4) 
modelina user behavior based on Baveslan Markov Chain Monte Carlo estimation prpoedgr?- 
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52-72 (cancelled) 

73. (Previously presented) The system of Claim 24 wherein the communication management 
system Is operable to generate e set of prediction ailes for allocating treatments to users with 
specific behavtorah characteristics. 

74. (Previously presented) The system of Claim 73 wherein the content system is operable to 
allocate treatments to users according to the prediction rules. 

75. (Previously presented) The system of aaim 24 wherein the communication management 
system implements a personalization process. 

76. (currently amended) The method of claim 1S5 wherein the experiment is delivered over 
the data network to respective control groups of users acconding to the experiment and wherein, 
the method further comprises generating a model for predicUng anticipated behavior of users 
based upon the observed behavion and generating a prediction for delivering one of the 
treatments to users according to the model In order to optimize a desired outcome. 

77. (Previously presented) The method of Claim 76 wherein defining an experiment 
comprises identifying elements of content which potentially influence behavior of users. 

78. (cancelled) 

79. (cancelled) 
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SO. (currently amended) A machine readable medium including a set of instructions for 

definino an experiment to gauge user rea a tinn tn various treatments for a set of content 
elements: 

conducting the experiment over a da ta nebftfork: and 
mllectina over the data network observatio n data relating to user Ipeh^vior for each treatment 
^ method of C l aim 15 \ Arfnerein con ducting the experiment comprises : 

statisticaliv sampling to specify a plurality of c ontrol groups, each control group 
comprising at least one user 

specifying a particular treatment to be deiiv e red to the at least one user in e^ch control 

gppup: ... . . 

recelvino identical reouests for content from the resp ective at least one user in each of 

ffle control croups: and 

In response to the identical reouests. delivering to the at least one usei- in 

each control group a different tre afrnent ^norating prodiot i ono o f^-^^^^ 

uDoro who oro to bo prooontcd a trootmcnt comprioing a oot of content 
oiomcntp wil l roaot to tho troatmont, tho prcdictiono boing gcnoratod from 
• data col l ootcd from ono or more exporimonts that tooted uoor bcha'/ior to 
oontont olGmonto in tho oot: af>d 
d o H voring trcatmonto to uoorc in accordanoo w i th tho prodictions to i ncroooo the 
probab il ity of □ dcoinsd uoor behavior in rcoponGo to tho prooontcd 

81 . (currently amended) A content page comprising a set of content elements generated by 
a computer implement ed system fon 
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defining an experiment to Gauoe user reaction to v a rious treatments for a set of content 
conductinq the experiment over a data network: and 

nollectina over the data network observation data rel ating to user behavior for each treatment 
ThG method of Claim 4g-wherein- conducting the exp e riment comprises: 

statistically sampling to soecHv a plunalitv of control groups, each control group 
comprising at least one user. 

specifying a particular treatment to be del ivared to the at least one user in each control 

groups 

receiving identical reouests for content from t h e respective at 'east one user in each of 
the control oroups: and 

in response to the Identicat requests, deitve ri no to the at least one user in gach CQhtrol 
group a different treatmen t a prodiotion onglno. the prcdiotion cng i no gon e ratiR g 
fif cdict i ono boood on data co ll octod from ono or more ojcporimcntG that tpotcd usor 
bohavior in rooponoo to oontcnt olomonto in tho sot . 

I 82. (currently amended) . The system of claim 6424 wherein the designed experiment 
reduces the number of combinations included in the experiment by measuring and 
estimating only those effects that are of a predetermined Interest 

83. (new) An automated system for experimentaton. the system being configured to: 

generate and allocate a set of treatments from a predefined set of content elements, 

according to experiment designs, the system capable of deterrrtining a design by 
selecting from design .criteria comprising: the specific effects allowed; the 
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number of factors and factor levels included: and the amount of information 
produced In relation to the objective outcome; and 
collect obseivation data of user behavior to the set of treatments and determine the 
effect of each content element on user behavior according to choice models, 
■ behavioral models.- or contingency tables. 

84. (new) The system of daim 83 wherein the system is further configured to detemnine and 
allocate a treatment from the set of content elements to increase the probability of achieving a 
desired objective based on the determination of the effects of the content elements. 

85. (new) A computer Implemented method bomprising: 

automatically creating various treatments for a set of content elements according to 
systematic atles of experimentation that detemnine a number of alternative 
treatments using various combinations of the content elements fl-om the set; 

allocating the treatments to users to test user behavior in response to the various 
treatments; 

capturing data relating to the observed behavior of users relative to the various 
treatments; 

performing one or more of: (1) analyzing the data to derive a model in form of a 

contingency table; (2) processing the data for pathologies, including at least one , 
of missing data, and structural dependencies, and processing the data to be 
model ready: (3) making inferences about some variables that Influence user 
behavior based on random utility theory; and (4) modeling user behavior based 
on Bayeslan Mari^ov Chain Monte Cario estimation pnDoedure. 
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86. (new) A computer implemented method, comprising: 

generating and allocating a set of treatments from a predefined set of content elements, 
according to experiment designs, the system capable of detemiining a design by 
selecting from design criteria comprising: the specific effects allowed: the 
number of fatstors and factor-levels Included; and the amount of Information 
produced In relation to the objective outcome; and 

collecting observation data of user behavior for the set of treatments and detemiining the 
Sect of each content element on user behavior according to choice models, 
behavioral models, or contingency tables. 

87. (new) The method of claim 8B further comprising determining and allocating a treatment 

from the set of content elements to increase the probability of achieving a desired 
objective based on the determination of the effects of the content elements. 

88. (new) A machine readable medium storing a set of instmctlons for performing the steps 

comprising: 

automatically creating various treatments for a set of content elements according to 
systematic rules of experimentation that determine a number of alternative 
treatments using various combinations of the content elements from the set; 

allocating the treatments to users to test user behaviorln response to the various 
treatments 

capturing data relating to the observed behavior of users relative to the various 
treatments; 

performing one or more of: (1 ) analyang the data to derive a model In fomn of a 
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of missing data, and structural dependencies, and processing the data to be 
model ready; (3) making inferences about some variables that influence user 
behavior based on random utility theory; and (4) modeling user behavior based 
on Bayesian Markov Chain Monte Carlo estimation procedure. 

89. (new) A machine readable medium storing a set of instoictions for perfonning the steps 
comprising: 

generating and allocating a set of treatments from a predefined set of content elements, 
according to experiment designs, the system capable of determining a design by 
selecting from design cfTterta comprising: the specific effects allowed; the 
number of factors and factor levels included; and the amount of information 
produced In relation to the objective outcome; and 

collecting observation data of user behavior for the set of treatments and detemnining the 
effect of each content element on user behavior according to choice models, 
behavioral models, or contingency tables. 

90. (new) The machine readable medium of claim 89 further comprising instructions for 
detennining and allocating a treatment from the set of content elements to increase the 
probability of achieving a desired objective based on the detemnination of the effects of 
the content elements. 

91 . (new) A computer implemented method; comprisirig: 

defining an experiment to gauge user reaction to various treatments for a set of content 
elements; 

conducting the experiment over a data network; and 
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collecting over the data network observation data relating -to user behavior for each treatment 
wherein conducting the experiment comprises: 

statistically sampling to specify a plurality of control groups, each control group 

comprisirvg at least one user^ 

- specifying a particular treatment to be delivered to the at least one user in each control 

group; 

receiving identical requests for content from the respective at least one user In each of 
the control groups; and 

in response to the Identical requests, delivering to the at least one user in each control 
group a different treatment 
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